Solid state storage, in particular, flash-based devices either in solid state drives (SSDs) or on flash cards, is quickly emerging as a credible tool for use in enterprise storage solutions. Ongoing technology developments have vastly improved performance and provided for advances in enterprise-class solid state reliability and endurance. As a result, solid state storage, specifically flash storage deployed in SSDs, are becoming vital for delivering higher performance to servers and storage systems, such as the data warehouse system illustrated in FIG. 1.
The system illustrated in FIG. 1, a product of Teradata Corporation, is a hybrid data warehousing platform that provides the capacity and cost benefits of hard disk drives (HDDs) while leveraging the performance advantage of solid-state drives (SSDs). As shown the system includes multiple physical processing nodes 101, connected together through a communication network 105. Each processing node may host one or more physical or virtual processing modules, such as one or more access module processors (AMPs). Each of the processing nodes 101 manages a portion of a database that is stored in a corresponding data storage facility including SSDs 120, providing fast storage and retrieval of high demand “hot” data, and HDDs 110, providing economical storage of lesser used “cold” data.
Teradata Virtual Storage (TVS) software 130 manages the different storage devices within the data warehouse, automatically migrating data to the appropriate device to match its temperature. TVS replaces traditional fixed assignment disk storage with a virtual connection of storage to data warehouse work units, referred to as AMPs within the Teradata data warehouse. FIG. 2 provides an illustration of allocation of data storage in a traditional Teradata Corporation data warehouse system, wherein each AMP owns the same number of specific disk drives and places its data on those drives without consideration of data characteristics or usage.
FIG. 3 provides an illustration of allocation of data storage in a Teradata Corporation data warehouse system utilizing Teradata Virtual Storage (TVS). Storage is owned by Teradata Virtual Storage and is allocated to AMPs in small pieces from a shared pool of disks. Data are automatically and transparently migrated within storage based on data temperature. Frequently used hot data is automatically migrated to the fastest storage resource. Cold data, on the other hand, is migrated to slower storage resources.
Teradata Virtual Storage allows a mixture of different storage mechanisms and capacities to be configured in an active data warehouse system. TVS blends the performance-oriented storage of small capacity drives with the low cost-per-unit of large capacity storage drives so that the data warehouse can transparently manage the workload profiles of data on the storage resources based on application of system resources to the usage.
Systems for managing the different storage devices within the data warehouse, such as TVS, are described in U.S. Pat. No. 7,562,195; and United States Patent Application Publication Number 2010-0306493, which are incorporated by reference herein.
Many of the prior systems for managing storage devices within a data warehouse were homogenous with respect to configured storage devices, e.g., HDDs, and allocations were placed on devices with the smallest queue depth. With the inclusion of SSDs and HDDs in the data warehouse, some of these systems may over-utilize the SSDs, while at the same time under utilize HDDs. A new method of answering allocation placement questions in systems comprised of different classes of SSD and HDD devices with devices shared amongst AMPs is desired. Described below is an improved system and method for allocating resources in a mixed SSD and HDD storage environment.